Endoscope system and control method

ABSTRACT

An endoscope system includes: a processor configured to perform image processing on endoscopic image data, the processor being connectable with multiple peripherals, the processor being configured to receive an input of a first audio command; a terminal device configured to wirelessly communicate with the processor and receive an input of a second audio command; a setting circuit configured to, in a case where the processor and the terminal device can communicate with each other, make a setting such that the first audio command that is receivable by the processor and the second audio command that is receivable by the terminal device differ from each other at least partly; and a first communication controller configured to, when the first audio command or the second audio command serves as an instruction to record the endoscopic image data, transmit the endoscopic image data to the terminal device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of PCT International Application No.PCT/JP2018/032421 filed on Aug. 31, 2018, which designates the UnitedStates, incorporated herein by reference, and which claims the benefitof priority from Japanese Patent Application No. 2017-238629, filed onDec. 13, 2017, incorporated herein by reference.

BACKGROUND 1. Technical Field

The disclosure relates to an endoscope system that displays image dataobtained by inserting an endoscope into a subject and capturing in-vivoimages of the subject and to a control method.

2. Related Art

As for recent endoscope systems, a technique to control multipleelectronic devices according to an audio input has been known (refer toJapanese Laid-open Patent Publication No. 2002-336184). In thetechnology, when control on the electronic devices is executed accordingto an audio command that is input via an audio input unit, notifyingabout a request for approval of the execution prevents an incorrectoperation caused by the audio input.

SUMMARY

In some embodiments, an endoscope system includes: processor configuredto perform image processing on endoscopic image data that is acquired byan endoscope that performs in-vivo observation on a subject, theprocessor being connectable with multiple peripherals, the processorbeing configured to receive an input of a first audio command serving asan instruction to drive any one of the peripherals based on audio datathat is generated by an audio input circuit receiving an input of sound;a terminal device configured to wirelessly communicate with theprocessor and receive an input of a second audio command serving as aninstruction to drive any one of the peripherals using sound; a settingcircuit configured to, in a case where the processor and the terminaldevice can communicate with each other, make a setting such that thefirst audio command that is receivable by the processor and the secondaudio command that is receivable by the terminal device differ from eachother at least partly; and a first communication controller configuredto, when the first audio command or the second audio command serves asan instruction to record the endoscopic image data, transmit theendoscopic image data to the terminal device.

In some embodiments, provided is a control method that is executed by anendoscope system including a processor configured to perform imageprocessing on endoscopic image data that is acquired by an endoscopethat performs in-vivo observation on a subject, the processor beingconnectable with multiple peripherals, the processor being configured toreceive an input of a first audio command serving as an instruction todrive any one of the peripherals based on audio data that is generatedby an audio input circuit receiving an input of sound; and a terminaldevice configured to wirelessly communicate with the processor andreceive an input of a second audio command serving as an instruction todrive any one of the peripherals using sound. The control methodincludes: in a case where the processor and the terminal device cancommunicate with each other, making a setting such that the first audiocommand that receivable by the processor and the second audio commandthat is receivable by the terminal device differ from each other atleast partly; and when the first audio command or the second audiocommand serves as an instruction to record the endoscopic image data,transmit the endoscopic image data to the terminal device.

The above and other features, advantages and technical and industrialsignificance of this disclosure will be better understood by reading thefollowing detailed description of presently preferred embodiments of thedisclosure, when considered in connection with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a functional configuration of anendoscope system according to a first embodiment of the disclosure;

FIG. 2 is a block diagram illustrating a functional configuration of aterminal device according to the first embodiment of the disclosure;

FIG. 3 is a flowchart illustrating an overview of a process that isexecuted by a processor according the first embodiment of thedisclosure;

FIG. 4 is a flowchart illustrating an overview of a connection processin FIG. 3;

FIG. 5 is a flowchart illustrating an overview of an audio input unitprocess in FIG. 3;

FIG. 6 is a flowchart illustrating an overview of a terminal deviceprocess in FIG. 3;

FIG. 7 is a flowchart illustrating an overview of a process that isexecuted by the terminal device according to the first embodiment of thedisclosure;

FIG. 8 is a flowchart illustrating an overview of a verification processin FIG. 7;

FIG. 9 is a flowchart illustrating an overview of a connection processin FIG. 7;

FIG. 10 is a block diagram illustrating a functional configuration of anendoscope system according to a second embodiment of the disclosure;

FIG. 11 is a block diagram illustrating a functional configuration of aterminal device according to a second embodiment of the disclosure;

FIG. 12 is a flowchart illustrating an overview of a process that isexecuted by a processor according to the second embodiment of thedisclosure; and

FIG. 13 is a flowchart illustrating a verification process that isexecute by the terminal device according to the second embodiment of thedisclosure.

DETAILED DESCRIPTION

An endoscope system including an endoscope that captures in-vivo imagesof an internal cavity of a subject, such as a patient, and displays thein-vivo images will be exemplified and described as a mode for carryingout the disclosure (“embodiment” below). The embodiment does not limitthe disclosure. In the description of the drawings, the same componentsare denoted by the same reference numbers.

First Embodiment

Configuration of Endoscope System

FIG. 1 is a block diagram illustrating a functional configuration of anendoscope system according to a first embodiment of the disclosure. Anendoscope system 1 illustrated in FIG. 1 is used when healthprofessionals including at least a doctor perform an endoscopic surgery,an endoscopic examination, or an endoscopic treatment on a subject, suchas a patient, in an operation room 100 in a hospital. The endoscopesystem 1 includes an endoscope 2, a processor 3, a wireless unit 4, aterminal device 5, a display device 6, a system controller 7, an audioinput unit 8, an ultrasound device 9, a pneumoperitoneum device 10, anelectrocautery device 11, a printer 12, a room light 13, an electronicoperating table 14, and a wireless power supply device 15.

First of all, a configuration of the endoscope 2 will be described. Theendoscope 2 is inserted into the subject. The endoscope 2 is configuredusing a rigid endoscope or a flexible endoscope. Under the control ofthe processor 3, the endoscope 2 applies illumination light to theinside of the subject, captures in-vivo images of an area of the subjectto which the illumination light is applied, generates endoscopic imagedata and then outputs the generated endoscopic image data to theprocessor 3. The endoscope 2 includes an imaging device 21 that capturesin-vivo images of the subject and generates image data. The imagingdevice 21 is configured using an image sensor, such as a charge coupleddevice (CCD) or a complementary metal oxide semiconductor (CMOS), an A/Cconversion circuit, etc. The endoscope 2 is connected to the processor 3in a wireless or wired manner such that interactive communication can beperformed in between. When transmitting the generated endoscopic imagedata wirelessly, the endoscope 2 may transmit the endoscopic image datasequentially to the processor 3 via the wireless unit 4 to be describedbelow or may transmit the endoscopic image data sequentially to a server200 that is set in the hospital and outside the operation room 100 via anetwork N100.

A configuration of processor 3 will be described. The processor 3controls the endoscope 2, performs given processing on the endoscopeimage data that is sequentially input from the endoscope 2, sequentiallyoutputs the processed endoscopic image data, and is connectable tomultiple peripherals and is capable of, based on audio data that isgenerated by the audio input unit 8 that receives an input of sound,receiving an input of a first audio command giving an instruction todrive any one of the peripherals. The first audio command is configuredof a command that makes it possible to operate all the peripherals. Theprocessor 3 includes a video processor 31, a communication unit 32, arecorder 33, an operation unit 34, an acquisition unit 35, and aprocessor controller 36.

The video processor 31 performs given image processing on the endoscopicimage data that is input from the endoscope 2 and outputs the processedendoscopic image data to the display device 6. The given imageprocessing includes synchronization processing, demosaicing processing(when the imaging device 21 has a Bayer array), white balance adjustmentprocessing, γ correction processing, chroma adjustment processing, andformat conversion processing. The video processor 31 is configured usinga field programmable gate array (FPGA), an application specificintegrated circuit (ASIC), a graphics processing unit (GPU), etc.

The communication unit 32 is configured using a communication module andperforms interactive communication with the terminal device 5 accordingto given communication standards. The communication unit 32 performsinteractive communication with the terminal device 5 via the wirelessunit 4 or performs interactive communication with the server 200 that isset in the hospital via the network N100. The given communicationstandards are Wi-Fi (Wireless Fidelity) (trademark) communication,Bluetooth (trademark) communication, Bluetooth Low Energy (trademark)communication (simply referred to as “BLE communication” below), or thelike. For example, Wi-Fi assumes a local area network and, as for rolesof devices, there is a relationship between an access point and astation and, as for a schematic connection process, there is arelationship in which the station is connected to a wireless networkthat is formed by the access point. In a rough connection sequence,first of all, the wireless unit 4 serving as an access point forms thewireless network and informs about a network identifier (SSID) of thenetwork. Sequentially, the communication unit 32 of the processor 3serving as the station searches for the network identifier (SSID) thatis informed about and connects to a desired network (access point). Anetwork with a large number of devices is assumed and the covered areawide and thus a strict identifying step is taken in consideration of theproblem of interference. This may increase the time to establishconnection. Note that, in data communication, it is possible to transmitor receive data at the timing of each of the access point and thestation. The communication unit 32 may employ communication by 4Gwireless communication other than Wi-Fi communication. Needless to say,the communication unit 32 may employ other communication, such ascommunication by 3G wireless communication, communication by 5G wirelesscommunication, WiMax (Worldwide interoperability for Microwave Access)(trademark) communication, or infrared communication (IrDA (InfraredData Association) (trademark)).

The recorder 33 records various programs to be executed by the processor3, data being processed, the endoscopic image data, etc. The recorder 33is configured using a flash memory, a synchronous dynamic random accessmemory (SDRAM), a memory card and a solid state drive (SSD), or thelike. The recorder 33 includes a verification information recorder 331that records a device address of a device authorized to performinteractive wireless communication with the processor 3 andconnectability determination information, a processor IP addressrecorder 332 that records a processor IP address that identifies theprocessor 3, and an audio command information recorder 333 that recordsa template for recognizing the control content of an audio command basedon audio data that is generated by the audio input unit 8. Theverification information recorder 331 records multiple sets of operatoridentification information that identify operators of the endoscope 2(“operator IDs” below), respectively, in association with levelsaccording to which operating each of the peripherals is permitted.

The operation unit 34 receives inputs of various types of information onthe endoscope system 1 and outputs the received information to theprocessor controller 36. The operation unit 34 is configured using, forexample, switches, buttons, a touch panel, etc. The operation unit 34receives an input of an operator ID that identifies a practitioner whoperforms treatment on the subject and outputs the operator ID to theprocessor controller 36.

The acquisition unit 35 acquires an operator ID that is set in an IDcard of an operator of the endoscope 2 and outputs the operator ID tothe processor controller 36. The operator ID includes an operator levelrepresenting an ID information level, a post, and a name. The operatorlevel is information representing permission to operate each of theperipherals according to each level. For example, an operator level ofan operating surgeon who is the operator (practitioner) is permitted tooperate all the peripherals, an operator level of a doctor who is anassistant is permitted to operate part of the peripherals, for example,excluding the ultrasound device 9, the electrocautery device 11 and theelectronic operating table 14, an operator level of an anesthesiologistis permitted to operate the pneumoperitoneum device 10, the printer 12,the room light 13 and the electronic operating table 14, an operatorlevel of a scrub nurse is permitted to operate the printer 12 and theroom light 13, and an operator level of a circulation nurse is permittedto operate the printer 12. The acquisition unit 35 is configured using,for example, a RFID reader or a communication module capable ofBluetooth communication.

The processor controller 36 controls each unit of the processor 3 andeach device configuring the endoscope system 1. The processor controller36 is configured using a central processing unit (CPU), etc. Theprocessor controller 36 includes a connection determination unit 361, adisplay controller 362, a communication controller 363, a setting unit364, a drive controller 365, an audio recognition unit 366, and aselector 367.

The connection determination unit 361 determines whether the terminaldevice 5 is a connection partner with which interactive communicationcan be performed based on a terminal device IP address and a result ofverification that are transmitted from the terminal device 5.

The display controller 362 controls the display mode of the displaydevice 6. Specifically, the display controller 362 causes the displaydevice 6 to display the endoscopic image corresponding to the endoscopicimage data on which the video processor 31 has performed the imageprocessing. When interactive wireless communication between theprocessor 3 and the terminal device 5 is established, the displaycontroller 362 causes the display device 6 to display informationindicating that the processor 3 and the terminal device 5 can performinteractive wireless communication.

Based on the result of determination performed by the connectiondetermination unit 361 and the result of verification that istransmitted from the terminal device 5, the communication controller 363permits the terminal device 5 to communicate with the peripherals.

Based on the level that is assigned to a registered user correspondingto the result of verification that is transmitted from the terminaldevice 5 and the verification information that is recorded by theverification information recorder 331, the setting unit 364 setsmultiple peripherals that are operable according to a second audiocommand of which input is received by the terminal device 5 via thesystem controller 7. Based on an operator ID that is acquired by theacquisition unit 35 and the verification information that is recorded inthe verification information recorder 331, the setting unit 364 furthersets multiple peripherals that are operable according to a first audiocommand of which input is received by the audio input unit 8 via thesystem controller 7.

Based on an audio command that is recognized by the audio recognitionunit 366 to be described below or a request signal and an operationsignal that are input from the terminal device 5 via the communicationunit 32, the drive controller 365 controls the system controller 7 andthereby control driving the peripherals. When the setting unit 364 hasnot make a setting such that peripherals corresponding to the secondaudio command that is input from the terminal device 5 are operable, thedrive controller disables the second audio command and does not executecontrol on the peripherals.

The audio recognition unit 366 refers to audio command information thatis recorded in the audio command information recorder 333 and recognizesthe content of an audio command based on audio data that is input fromthe audio input unit 8. Specifically, using the template representing awaveform of each audio command that is contained in the audio commandinformation that is recorded in the audio command information recorder333, the audio recognition unit 366 performs known pattern matching onthe audio data and thus recognizes the content of an audio command(first audio command) representing driving any one of the peripheralsand the control content and outputs the result of recognition to thedrive controller 365. Based on the audio data and based on thevoiceprints or feature values of the operator IDs that are registeredpreviously, the audio recognition unit 366 may recognize and specify theoperator who has input the sound to the audio input unit 8. The audiorecognition unit 366 may recognize the audio command content withrespect to audio data that is transmitted from the terminal device 5 viathe communication unit 32.

In the case where the processor 3 and the terminal device 5 cancommunicate with each other, when the processor 3 receives inputs of thefirst audio command via the audio input unit 8 and the second audiocommand from the terminal device 5, the selector 367 selects a priorityaudio command to be prioritized. Specifically, when the processor 3receives inputs of the first audio command and the second audio commandfrom the terminal device 5, the selector 367 selects a priority audiocommand to be prioritized from the first audio command and the secondaudio command based on the level of the operator that is acquired by theacquisition unit 35 and the level of the user who uses the terminaldevice 5.

A configuration of the wireless unit 4 will be described. The wirelessunit 4 is connected to the server 200 via the network N100 and isconnected to the processor 3 and the terminal device 5 according togiven communication standards such that the wireless unit 4 caninteractively communicate with the processor 3 and the terminal device5. The wireless unit 4 employs Wi-Fi communication. The wireless unit 4is set around the processor 3 or on a wall in the operation room 100.

A configuration of the terminal device 5 will be described. The terminaldevice interactively communicates with the processor 3 according togiven communication standards and receives the endoscopic image datathat is generated by the endoscope 2 and case image data from the server200 via the wireless unit 4 and displays the data. The terminal device 5is capable of receiving an input of the second audio command giving aninstruction to drive any one of the peripherals by sound. The secondaudio command is configured of a command that makes it possible tooperate a peripheral that does not directly relate to the operation. Thedetailed configuration of the terminal device 5 will be described below.

A configuration of the display device 6 will be described. Under thecontrol of the display controller 362, the display device 6 displays theimage corresponding to the image data that is input from the videoprocessor 31 and various types of information on the endoscope system 1.The display device 6 is configured using a liquid crystal or organicelectro luminescence (EL) display monitor, a speaker that outputs soundto the outside, etc.

The system controller 7 is connected to the processor 3 in a wired orwireless manner and individually controls each of the audio input unit8, the ultrasound device 9, the pneumoperitoneum device 10, theelectrocautery device 11, the printer 12, the room light 13, theelectronic operating table 14, and the wireless power supply device 15according to an instruction signal that is input from the processor 3.Any one of the audio input unit 8, the ultrasound device 9, thepneumoperitoneum device 10, the electrocautery device 11, the printer12, the room light 13, the electronic operating table 14, and thewireless power supply device 15 to be referred to is simply referred toas a “peripheral”. The system controller 7 is connected to each of theperipherals in a wired or wireless manner. The system controller 7 isconfigured using the CPU, a flash memory, etc.

Under the control of the system controller 7, the audio input unit 8picks up sound that is output from a sound source or a speaker, convertsthe sound into an analog audio signal (electric signal), performs A/Dconversion processing and gain adjustment processing on the audiosignal, generates digital audio data, and outputs the digital audio datato the processor 3 via the system controller 7. The audio input unit 8is configured using any one of a unidirectional microphone, anomnidirectional microphone and a bidirectional microphone, an A/Dconversion circuit and a signal processing circuit. The audio input unit8 is worn on the operator of the endoscope. Needless to say, the audioinput unit 8 may be arranged in the operation room 100 other than beingworn on the operator or may be worn on a nurse or an assistant otherthan the operator. The single audio input unit 8 need not be used andthe number of audio input units may be changed as appropriate. Forexample, audio input units corresponding in number to the healthprofessionals who are in the operation room 100 including the operatormay be arranged.

The ultrasound device 9 is connected to the endoscope 2 and, under thecontrol of the system controller 7, transmits and receives ultrasoundvia an ultrasound transducer that is arranged at the distal end of theendoscope 2. The ultrasound device 9 outputs ultrasound image data basedon the ultrasound that is received via the endoscope 2 to the systemcontroller 7. The ultrasound device 9 may generate ultrasound image dataon the subject via a dedicated ultrasound probe.

Under the control of the system controller 7, the pneumoperitoneumdevice 10 sends a pneumoperitoneum gas, such as carbon dioxide, into thesubject.

Under the control of the system controller 7, the electrocautery device11 drives the electrocautery by applying a given voltage to theelectrocautery.

Under the control of the system controller 7, the printer 12 outputs animage corresponding to the image data that is input from the processor3.

The room light 13 includes a plurality of room lights that are arrangedin the operation room 100 and, under the control of the systemcontroller 7, applies light to the subject and the operation room 100 ina given illuminance. The room light 13 is configured using a lightemitting diode (LED) lamp, a light adjustment switch, etc.

As for the electronic operating table 14, the subject is arranged on anoperating table. Under the control of the system controller 7, theelectronic operating table 14 is moved vertically and horizontally tochange the position and posture of the subject. The electronic operatingtable 14 is configured using an operating table that is movablevertically and horizontally and a driver, such as a motor, that drivesthe operating table.

Under the control of the system controller 7, the wireless power supplydevice 15 wirelessly supplies power to the terminal device 5. Thewireless power supply device 15 is configured using any one of anelectromagnetic induction system, a magnetic resonance system, anelectric field coupling system, and a radio transmission receptionsystem.

The server 200 is set outside the operation room 100 and in the hospitaland records the endoscopic image data that is transmitted from theprocessor 3 or the terminal device 5 via the network N100 a patient IDthat identifies the patient in association with each other. On receivingan image request signal requesting case image data and endoscopic imagedata via the network N100 or the wireless unit 4, the server 200transmits the case image data and the endoscopic image data to theprocessor 3 or the terminal device 5 that transmits the image requestsignal. The endoscopic image data herein includes video data and stillimage data (captured image data).

Configuration of Terminal Device

The detailed configuration of the terminal device referred to in FIG. 1will be described. FIG. 2 is a block diagram illustrating a functionalconfiguration of the terminal device 5.

The terminal device 5 illustrated in FIG. 2 includes a battery unit 50,a communication unit 51, an imaging unit 52, a fingerprint informationdetector 53, an audio input unit 54, a display unit 55, a recorder 56,an operation unit 57, and a terminal device controller 58.

The battery unit 50 includes a battery 501 that supplies power to eachunit configuring the terminal device 5 and a receiver 502 that receiveselectromagnetic waves that are supplied from the wireless power supplydevice 15, converts the electromagnetic waves into a current, andsupplies the current to the battery 501.

The communication unit 51 is configured using a communication module andperforms interactive communication with the processor 3 according to thegiven communication standards. The communication unit 51 interactivelycommunicates with the server 200 via the wireless unit 4 and the networkN100 in the hospital. As the given communication standards, Wi-Ficommunication is assumed. The communication unit 51 may employcommunication by 4G wireless communication other than Wi-Ficommunication. Needless to say, the communication unit 51 may employanother type of communication, such as Bluetooth communication, BLEcommunication, communication by 30 wireless communication, communicationby 5G wireless communication, WiMAX communication or infraredcommunication.

Under the control of the terminal device controller 58, the imaging unit52 captures an image of the user of the terminal device 5 and generatesimage data and then outputs the image data to the terminal devicecontroller 58. The imaging unit 52 is configured using an image sensor,such as a CCD or a CMOS, an image processing engine that is implementedusing A/D conversion processing, a FPGA or a GPU, etc. An infrared lampcapable of emitting infrared light and an image sensor in which pixelscapable of capturing images of the infrared light that is applied by theinfrared lamp may be arranged in the imaging unit 52 to acquireirregularities on the surface of the face of the user.

The fingerprint information detector 53 detects fingerprint informationon the fingers of the user that externally touches the fingerprintinformation detector 53 and outputs the result of detection to theterminal device controller 58. The fingerprint information detector 53configured using a fingerprint sensor. The fingerprint informationdetector 53 is configured using a fingerprint sensor. The fingerprintinformation detector 53 may employ a slide system other than a pressingsystem. The fingerprint information detector 53 may detect veins of theuser other than fingerprints.

Under the control of the terminal device controller 58, the audio inputunit 54 picks up the sound that is output from a sound source or aspeaker, converts the sound into an analog audio signal (electricsignal), performs A/D conversion processing and gain adjustmentprocessing on the audio signal, generates digital audio data, andoutputs the digital audio data to the terminal device controller 58. Theaudio input unit 54 is configured using any one of a unidirectionalmicrophone, an omnidirectional microphone and a bidirectionalmicrophone, an A/D conversion circuit, a signal processing circuit, etc.

The display unit 55 displays image data that is input from the terminaldevice controller 58 and various types of information. The display unit55 is configured using a liquid crystal or organic EL display panel.

The recorder 56 records various programs to be executed by the terminaldevice 5, data being processed, and image data. The recorder 56 isconfigured using a flash memory, a SSD, a memory card, etc. The recorder56 includes a verification information recorder 561, a terminal deviceIP address recorder 562 that records a terminal device IP address thatidentifies the terminal device 5, and an audio command informationrecorder 563 that records a template for recognizing the control contentof an audio command based on audio data that is input from the audioinput unit 54.

The operation unit 57 receives an input of an instruction signalcorresponding to an operation from the user. The operation unit 57 isconfigured using a touch panel, buttons, switches, etc.

The terminal device controller 58 generally controls the components ofthe terminal device 5. The terminal device controller 58 analyzes theaudio data that is input from the audio input unit 54, generates anaudio command based on the result of the analysis, and transmits theaudio command to the processor 3. The terminal device controller 58 isconfigured using the CPU, etc. The terminal device controller 58includes a connection determination unit 581, a verification unit 582, acommunication controller 583, a display controller 584, a recordingcontroller 585, an imaging controller 586, and an audio recognition unit587.

Based on the verification information that is received by thecommunication unit 51 from the processor 3, the connection determinationunit 581 determines whether the processor 3 is a connection partner withwhich interactive wireless communication can be performed.

The verification unit 582 verifies whether the user of the terminaldevice 5 is the registered user who is registered previously.Specifically, the verification unit 582 performs verification in whichany one of a face image of the user of the terminal device 5, biologicalinformation of the user, and gesture information on the user isacquired. For example, the verification unit 582 determines whether thefeature of the face image of the user in the image corresponding to theimage data that is generated by the imaging unit 52 matches the featureof the registered user that is recorded in the recorder 56.

Based on the result of determination by the connection determinationunit 581 and the result of verification by the verification unit 582,the communication controller 583 permits interactive wirelesscommunication between the processor 3 and the terminal device 5 orinteractive wireless communication between the network N100 and theterminal device 5.

The display controller 584 controls the display mode of the display unit55. Specifically, the display controller 584 causes the display unit 55to display an endoscopic image corresponding to endoscopic image dataand a case image corresponding to case image data.

The recording controller 585 causes the recorder 56 to record a patientround time contained in schedule information that is acquired by thecommunication controller 583 and the endoscopic image data inassociation with each other.

When the verification unit 582 verifies that the user is the registereduser, the imaging controller 586 enables the imaging function of theimaging unit 52 by the user of the terminal device 5.

The audio recognition unit 587 refers to the audio command informationthat is recorded in the audio command information recorder 563 andrecognizes the content of the audio command based on the audio data thatis input from the audio input unit 54. Specifically, using the templaterepresenting a waveform of each audio command contained in the audiocommand information that is recorded in the audio command informationrecorder 563, the audio recognition unit 587 performs known patternmatching on the audio data and thus recognizes the content of an audiocommand representing driving any one of the peripherals and the controlcontent and outputs the result of recognition to the communicationcontroller 583.

Process Executed by Processor

A process that is executed by the processor 3 will be described. FIG. 3is a flowchart illustrating an overview of the process that is executedby the processor 3.

As illustrated in FIG. 3, first of all, the processor 3 executes aconnection process of establishing a connection for interactivelycommunicating with the terminal device 5 (step S101). After step S101,the processor 3 moves to step S102 to be described below.

Connection Process

An overview of the connection process that is referred to at step S101in FIG. 3 will be described. FIG. 4 is a flowchart illustrating anoverview of the connection process.

As illustrated in FIG. 4, first of all, the communication controller 363transmits the processor IP address (SSID) and the verificationinformation to the terminal device 5 via the communication unit 32 (stepS201). The verification information is information for requesting aresult of verification performed by the verification unit 582, which isa verification result functioning as a password for interactive wirelesscommunication of the terminal device 5.

Sequentially, on receiving the terminal device IP address and the resultof verification performed by the verification unit 582 from the terminaldevice 5 via the communication unit 32 (YES at step S222), the processor3 moves to step S203 to be described below. On the other hand, when theterminal device IP address and the result of verification performed bythe verification unit 582 are not received from the terminal device 5via the communication unit 32 (NO at step S202), the processor 3 movesto step S205.

At step S203, based on the terminal device IP address and the result ofverification performed by the verification unit 582 that are received bythe communication unit 32, the connection determination unit 361determines whether the terminal device 5 is a connection partner withwhich interactive wireless communication can be performed (step S203).Specifically, the connection determination unit 361 determines whetherthe terminal device IP address and the result of verification performedby the verification unit 582 that are received by the communication unit32 match the verification information that is recorded in theverification information recorder 331. While it is determined that theterminal device 5 is a connection partner with which interactivewireless communication can be performed when the terminal device IPaddress and the verification result match the verification information,it is determined that the terminal device 5 is not a connection partnerwith which interactive wireless communication can be performed when theterminal device IP address and the verification result do not match theverification information. When the connection determination unit 361determines that the terminal device 5 is a connection partner with whichinteractive wireless communication can be performed (YES at step S203),the processor 3 moves to step S204 to be described below. On the otherhand, when the connection determination unit 361 determines that theterminal device 5 is not a connection partner with which interactivewireless communication can be performed (NO at step S203), the processor3 moves to step S205 to be described below.

At step S204, the communication controller 363 connects the processor 3and the terminal device 5 such that the processor 3 and the terminaldevice 5 can interactively communicate with each other. Accordingly, theprocessor 3 enters a state where the processor 3 can interactivelycommunicate with the terminal device 5. In this case, the displaycontroller 362 may cause the display device 6 to display thatinteractive wireless communication can be performed between theprocessor 3 and the terminal device 5. In other words, the displaycontroller 362 functions as an informing unit. After step S204, theprocessor 3 returns to the main routine in FIG. 3.

At step S205, the display controller 362 causes the display device 6 todisplay a warning that the terminal device 5 is not a connection partnerwith which interactive wireless communication can be performed. Thedisplay controller 362 causes the display device 6 display a warningthat the terminal device 5 is not a connection partner with whichinteractive wireless communication can be performed; however,alternatively, for example, a warning that the terminal device 5 is nota connection partner with which interactive wireless communication canbe performed may be informed about using a speaker, which is notillustrated in the drawings, or the like. In other words, the displaycontroller 362 functions as an informing unit that informs thatinteractive wireless communication between the processor 3 and theterminal device 5 cannot be performed. After step S205, the processor 3returns to the main routine in FIG. 3.

FIG. 3 will be referred back to continue describing step S102 and thefollowing steps.

At step S102, when the processor 3 and the terminal device 5 areinteractively communicating with each other (YES at step S102), theacquisition unit 35 acquires an operator ID (step S103).

Sequentially, based on an operator ID level that is acquired by theacquisition unit 35 and the level that is assigned to the registereduser corresponding to the verification result that is received from theterminal device 5, the setting unit 364 sets communication connectionwith each of multiple peripherals that are operable according to anaudio command of which input is received by each of the audio input unit8 and the terminal device 5 via the system controller 7 (step S104).Specifically, when the operator ID level and the level of the registereduser are the level of doctor, the setting unit 364 makes a setting suchthat all the peripherals are operable via the terminal device 5 areoperable. On the other hand, when the operator ID level and the level ofthe registered user are the level of nurse, the setting unit 364 setsonly given peripherals for peripherals operable via the terminal device5, for example, makes a setting such that peripherals not related to theoperation, more specifically, the printer 12, the room light 13, thewireless power supply device 15, etc., are operable. Needless to say,based on the operator ID level and the level of the registered user, thesetting unit 364 may finely set peripherals operable via the audio inputunit 8 and the terminal device 5.

Sequentially, in the case where an audio command comes from the audioinput unit 8 or the terminal device 5 (YES at step S105), when there aremultiple commands within a given time (for example, one to two seconds)(YES at step S106), the selector 367 selects an audio command to beprioritized based on the levels (step S107). Specifically, the selector367 compares the operator ID level of the audio input unit 8 and thelevel of the registered user and preferentially selects the audiocommand of the higher level.

Sequentially, when an audio command is input from the audio input unit 8(YES at step S108), the processor 3 moves to step S109 to be describedbelow. On the other hand, when no audio command is input from the audioinput unit 8 (NO at step S108), the processor 3 moves to step S110 to bedescribed below.

At step S109, the processor 3 executes an audio input unit process ofdriving peripherals according to the audio command that is input fromthe audio input unit 8. After step S109, the processor 3 moves to stepS111 to be described below.

Audio Input Unit Process

An overview of the audio input unit process referred to at step S109 inFIG. 3 will be described. FIG. 5 is a flowchart illustrating an overviewof the audio input unit process.

As illustrated in FIG. 5, first of all, when the audio command that isinput from the audio input unit 8 serves as an instruction to record theendoscopic image data (YES at step S301), the communication controller363 transmits the endoscopic image data to the terminal device 5 via thecommunication unit 32 (step S302). After step S302, the processor 3moves to step S303 described below.

At step S301, when the audio command that is input from the audio inputunit 8 does not serve as an instruction to record the endoscopic imagedata (NO at step S301), the processor 3 moves to step S303 to bedescribed below.

At step S303, when a peripheral to be controlled according to the audiocommand that is input from the audio input unit 8 is a controllableperipheral that is set by the setting unit 364 (YES at step S303), thedrive controller 355 controls the peripheral according to the controlcontent corresponding to the audio command that is recognized by theaudio recognition unit 366 (step S304).

Sequentially, the communication controller 363 transmits tag informationon the content of the audio command and the endoscopic image data inassociation with each other to the terminal device 5 via thecommunication unit 32 (step S305). After step S305, the processor 3returns to the main routine in FIG. 3.

At step S303, when the peripheral that is controlled according to theaudio command that is input from the audio input unit 8 is not acontrollable peripheral that is set by the setting unit 364 (NO at stepS303), the display controller 362 causes the display device 6 to displaya warning that the peripheral is uncontrollable (step S306). After stepS306, the processor 3 returns to the main routine in FIG. 3.

FIG. 3 will be referred back to continue describing step S110 and thefollowing steps.

At step S110, the processor 3 executes a terminal device process ofdriving a peripheral according to an audio command that is input fromthe terminal device 5. After step S110, the processor 3 moves to stepS111 to be described below.

Terminal Device Process

An overview of the terminal device process referred to at step S110 inFIG. 3 will be described. FIG. 6 is a flowchart illustrating an overviewof the terminal device process.

As illustrated in FIG. 6, first of all, when the audio command that isinput from the terminal device 5 serves as an instruction to transmitthe endoscopic image data (YES at step S401), the communicationcontroller 363 transmits the endoscopic image data to the terminaldevice 5 via the communication unit 32 (step S402). After step S402, theprocessor 3 moves to step S403 to be described below.

At step S402, when the audio command that is input from the terminaldevice 5 does not serve as an instruction to transmit the endoscopicimage data (NO at step S401), the processor 3 moves to step S403 to bedescribed below.

At step S403, when a peripheral to be controlled according to the audiocommand that is input from the terminal device 5 is a controllableperipheral that is set by the setting unit 364 (YES at step S403), thedrive controller 365 controls the peripheral according to the controlcontent corresponding to the audio command that is recognized by theaudio recognition unit 366 (step S404).

Sequentially, the communication controller 363 transmits tag informationon the content of the audio command and the endoscopic image data inassociation with each other to the terminal device 5 via thecommunication unit 32 (step S405). After step S405, the processor 3returns to the main routine in FIG. 3.

At step S403, when the peripheral that is controlled. according to theaudio command that is input from the terminal device 5 is not acontrollable peripheral that is set by the setting unit 364 (NO at stepS403), the communication controller 363 transmits an uncontrollabilitysignal indicating that the peripheral is not operable to the terminaldevice 5 via the communication unit 32 (step S406). After step S406, theprocessor 3 returns to the main routine in FIG. 3.

FIG. 3 will be referred back to continue describing step S111 and thefollowing steps.

At step S111, when an instruction signal of an instruction to end isinput from the operation unit 34 or the terminal device 5 (YES at stepS111), the processor 3 moves to step S112 to be described below. On theother hand, when no instruction signal of an instruction to end is inputfrom the operation unit 34 or the terminal device 5 (NO at step S111),the processor 3 returns to step S105 described above.

At step S102, when the processor 3 and the terminal device 5 are notcommunicating with each other (NO at step S102), the processor ends theprocess.

At step S105, when no audio command comes from the audio input unit 8 orthe terminal device 5 (NO at step S105), the processor 3 moves to stepS111.

At step S106, when there are not multiple commands within the given time(NO at step S106), the processor 3 moves to step S108.

At step S112, the communication controller 363 records the terminaldevice IP address of the terminal device 5 in the recorder 33. Thispromptly enables communication between the processor 3 and the terminaldevice 5 when the terminal device 5 is powered on in the operation room100. After step S112, the processor 3 ends the process.

Process executed by Terminal Device

A process that is executed by the terminal device 5 will be described.FIG. 7 is a flowchart illustrating an overview of the process that isexecuted by the terminal device 5.

As illustrated in FIG. 7, first of all, the terminal device 5 executes averification process of verifying whether the user of the terminaldevice 5 is the registered user who can operate the terminal device 5(step S701). After step S701, the terminal device 5 moves to step S702to be described below.

Verification Process

Details of the verification process referred to at step S701 in FIG. 7will be described. FIG. 8 is a flowchart illustrating an overview of theverification process.

As illustrated in FIG. 8, the verification unit 582 acquires informationon the user of the terminal device 5 (step S801). Specifically, theverification unit 582 acquires any one of image data obtained by theimaging unit 52 by capturing an image of the user of the terminal device5, fingerprint information on the user that is detected by thefingerprint information detector 53, and audio data that is generated bythe audio input unit 54 from a corresponding one of the imaging unit 52,the fingerprint information detector 53, and the audio input unit 54.

Sequentially, based on information on the user of the terminal device 5and the verification information that is recorded in the verificationinformation recorder 561, the verification unit 582 determines whetherthe user is the registered user (step S802). Specifically, based on theimage data that is generated by the imaging unit 52 and verificationinformation that is recorded in the verification information recorder561, the verification unit 582 verifies whether the user is theregistered user. More specifically, the verification unit 582 determineswhether the feature of the face of the user in the image correspondingto the image data that is generated by the imaging unit 52 and thefeature of the face of the registered user contained in the verificationinformation that is recorded in the verification information recorder561 have a given matching degree. While the verification unit 582verifies that the user is the registered user when the features have thegiven matching degree, and the verification unit 582 verifies that theuser is not the registered user when the features do not have the givenmatching degree. The verification unit 582 verifies whether the user isthe registered user based on the fingerprint information that isdetected by the fingerprint information detector 53 and the verificationinformation that is recorded in the verification information recorder561. More specifically, the verification unit 582 determines whether thefeature of the fingerprint information on the user that is detected bythe fingerprint information detector 53 and the feature of thefingerprint information on the registered user contained in theverification information that is recorded in the verificationinformation recorder 561 have a given matching degree. While theverification unit 582 verifies that the user is the registered user whenthe features have the given matching degree, the verification unit 582verifies that the user is not the registered user when the features donot have the given matching degree. The verification unit 582 furtherverifies whether the user is the registered user based on the audio datathat is generated by the audio input unit 54 and the verificationinformation that is recorded in the verification information recorder561. More specifically, the verification unit 582 determines whether thefeature of the voice print of the audio data that is generated by theaudio input unit 54 and the feature of the voice print of the registereduser contained in the verification information that is recorded in theverification information recorder 561 have a given matching degree.While the verification unit 582 verifies that the user is the registereduser when the features have the given matching degree, the verificationunit 582 verifies that the user is not the registered user when thefeatures do not have the given matching degree. The verification unit582 determines whether the gesture information on the user in the imagecorresponding to the image data that is generated by the imaging unit 52and the gesture information contained in the verification informationthat is recorded in the verification information recorder 561 have agiven matching degree. While the verification unit 582 verifies that theuser is the registered user when the sets of gesture information havethe given matching degree, the verification unit 582 verifies that theuser is not the registered user when the sets of gesture information donot have the given matching degree. The gesture information hereinrefers to, for example, the number of blinks of the user, the shape ofthe mouth, the moves of the face, the moves of the hands, etc. When theverification unit 582 determines that the user of the terminal device 5is the registered user (YES at step S802), the verification unit 582permits the user to use the terminal device 5 (step S803).

The display controller 584 causes the display unit 55 to display a GUIcapable of receiving operational inputs of the user, or the like (stepS804). While the display controller 584 causes the display unit 55 todisplay a GUI capable of receiving operational inputs of the user, orthe like; however, alternatively, the display controller 584 may informthat use of the terminal device 5 is permitted using a speaker notillustrated in the drawings, or the like. In other words, the displaycontroller 584 functions as an informing unit that informs thatinteractive wireless communication between the processor 3 and theterminal device 5 or interactive communication between the network N100and the terminal device 5 can be performed. After step S804, theterminal device 5 returns to the above-described main routine in FIG. 7.

At step S802, when the verification unit 582 determines that the user ofthe terminal device 5 is not the registered user (NO at step S802), theverification unit 582 does not permit the user to use the terminaldevice 5 (step S805).

The display controller 584 keeps the display unit 55 from displaying orcauses the display unit 55 to display a warning (step S806). The displaycontroller 584 keeps the display unit 55 from displaying or causes thedisplay unit 55 to display a warning; however, alternatively, a warningmay be output using a speaker not illustrated in the drawings, or thelike. After step S806, the terminal device 5 returns to theabove-described main routing in FIG. 7.

FIG. 7 will be referred back to continue describing step S702 and thefollowing steps.

At step S702, the terminal device 5 executes a connection process forperforming interactive communication with the processor 3 (step S702).After step S702, the terminal device 5 moves to step S703 to bedescribed below.

Connection Process

Details of the connection process referred to at step S702 in FIG. 7will be described. FIG. 9 is a flowchart illustrating details of theconnection process.

As illustrated in FIG. 9, first of all, the connection determinationunit 581 determines whether the processor IP address (SSID) and theverification information are received from the processor 3 via thecommunication unit 51 (step S901). The verification information hereinrefers to information for requesting the result of verificationperformed by the verification unit 582, which is a verification resultfunctioning as a password for interactive wireless communication withthe processor 3. When the processor IP address (SSID) is received fromthe processor 3, the display controller 584 may cause the display unit55 to display information on the processor IP address. In this case, enthe display unit 55 displays multiple addresses, the user preferablyselects a desired address via the operation unit 57. When the connectiondetermination unit 581 determines that the processor IP address and theverification information are received from the processor 3 via thecommunication unit 51 (YES at step S901), the terminal device 5 moves tostep S902 to be described below. On the other hand, when the connectiondetermination unit 581 determines that the processor IP address and theverification information are not received from the processor 3 via thecommunication unit 51 (NO at step S901), the terminal device 5 performsthis determination at given intervals (for example, every 10 seconds).

The communication controller 583 causes the communication unit 51 totransmit, to the processor 3, the terminal device IP address that isrecorded in the terminal device IP address recorder 562 that is recordedin the recorder 56 and the result of verification performed by theverification unit 582 (step S902). In this case, the communicationcontroller 583 causes the communication unit 51 to transmit the terminaldevice IP address and the verification result to an access point (theSSID of the processor 3 or the SSID of the wireless unit 4) that isselected previously by the user via the operation unit 57. In the firstembodiment, the access point is the processor 3 or the wireless unit 4.Alternatively, the terminal device 5 may serve as the access point. Inthis case, the communication controller 583 may transmit a terminaldevice IP address (SSID) indicating that the terminal device 5 is withina given area, such as a communication area of Wi-Fi, via thecommunication unit 51.

At step S903, the connection determination unit 581 determines whetherthe processor IP address that is received by the communication unit 51is a connection partner with which the terminal device 5 caninteractively communicate (step S903). Specifically, the connectiondetermination unit 581 determines whether the processor IP address is aprocessor IP address (SSID) that is selected by the user via theoperation unit 57. When the connection determination unit 581 determinesthat the processor IP address that is received by the communication unit51 is a connection partner with which the terminal device 5 cancommunicate interactively (YES at step S903), the terminal device 5moves to step S904 to be described below. On the other hand, when theconnection determination unit 581 determines that the processor IPaddress that is received by the communication unit 51 is not aconnection partner with which the terminal device 5 can communicateinteractively (NO at step S903), the terminal device 5 moves to stepS906 to be described below.

At step S904, the communication controller 583 connects the processor 3and the terminal device 5 such that the processor 3 and the terminaldevice 5 can communicate interactively. Accordingly, the terminal device5 enters a state where the terminal device 5 can interactivelycommunicate with the processor.

The display controller 584 causes the display unit 55 to displayconnection completion indicating that connecting with the processor 3has completed (step S905). Accordingly, the user is able toinstinctively know that the terminal device 5 enters a state where theterminal device 5 can interactively communicate with the processor 3.After step S905, the terminal device 5 returns to the main routine inFIG. 7. The display controller 584 causes the display unit 55 to displaythe connection completion; however, the embodiment is not limitedthereto. For example, a speaker (not illustrated in the drawings) may beused to make an output indicating that connecting with the processor 3has completed or a driver (not illustrated in the drawings), such as amotor, may be driven to inform that connecting with the processor 3 hascompleted.

At step S906, the display controller 584 causes the display unit 55 todisplay that it is not possible to connect with the processor 3. Thisallows the user to instinctively know that it is not possible to connectwith the processor 3. After step S906, the terminal device 5 returns tothe main routine in FIG. 7.

FIG. 7 will be referred back to continue describing step S703 and thefollowing steps.

At step S703, when the terminal device 5 and the processor 3 areinteractively communicating with each other (YES at step S703), theterminal device 5 moves to step S704. On the other hand, when theterminal device 5 and the processor 3 are not interactivelycommunicating with each other (No at step S703), the terminal device 5ends the process.

At step S704, when an audio command is input from the audio input unit54 (YES at step S704), the communication controller 583 transmits, tothe processor 3, the audio command that is input from the audio inputunit 54 via the communication unit 51 (step S705). After step S705, theterminal device 5 moves to step S706 to be described below. On thecontrary, when no audio command is input from the audio input unit 54(NO at step S704), the terminal device 5 ends the process.

At step S706, in the case where the endoscopic image data is receivedfrom the processor 3 (YES at step S706), when tag information on thecontrol content of the audio command is received from the processor 3(YES at step S707), the recording controller 585 records the endoscopicimage data and the tag information on the control content of the audiocommand in association with each other in the recorder 56 (step S708).

Based on the tag information that is received from the processor 3, thecommunication controller 583 acquires relevant image data from theserver 200 via the communication unit 51 and the network N100 (stepS709). The relevant image data herein refers to image data relevant tothe tag information that is, for example, case image data, such asendoscopic image data obtained by capturing images of another subjectafter execution of the audio command. In this case, the recordingcontroller 585 may cause the recorder 56 to record the relevant imagedata that is acquired by the communication controller 583 in associationwith the tag information and the endoscopic image data.

The display controller 584 then causes the display unit 55 to display anendoscopic image corresponding to the endoscopic image data and arelevant image corresponding to the relevant image data (step S710). Inthis case, when the endoscopic image data that is received by thecommunication unit 51 is video data that is sequentially transmitted,the display controller 584 sequentially updates the endoscopic imagecorresponding to the endoscopic image data chronologically. The displaycontroller 584 causes the display unit 55 to display the endoscopicimage and the relevant image such that the endoscopic image and therelevant image can be compared with each other. For example, the displaycontroller 584 may cause the display unit 55 to display the endoscopicimage and the relevant image in parallel, to make a display in which therelevant image that is reduced in size according to a given ratio issuperimposed onto the endoscopic image, or to make a display in whichthe endoscopic image that is reduced in size according to a given ratiois superimposed onto the relevant image.

The communication controller 583 transmits the endoscopic image data,the tag information, and the result of verification performed by theverification unit 582 in association with one another to the server 200via the communication unit 51 and the network N100 (step S711). Afterstep S711, the terminal device 5 moves to step S713 to be describedbelow.

At step S706, when the endoscopic image data is not received from theprocessor 3 (NO at step S706), the terminal device 5 moves to step S713to be described below.

At step S707, when the tag information on the control content of theaudio command is not received from the processor 3 (NO at step S707),the recording controller 585 records the endoscopic image data that isreceived from the processor 3 in the recorder 56 (step S712). After stepS712, the terminal device 5 moves to step S713 to be described below.

At step S713, when an instruction signal to end operations of theendoscope system 1 while communication with the processor 3 is inputfrom the operation unit 57 (YES at step S713), the terminal device 5moves to step S714 to be described below. On the other hand, when noinstruction signal to end operations of the endoscope system whilecommunicating with the processor 3 is input from the operation unit 57(NO at step S713), the terminal device 5 returns to step S704 describedabove.

At step S714, the recording controller 585 records the processor IPaddress of the processor 3 in the recorder 56. After step S714, theterminal device 5 ends the process.

According to the above-described first embodiment of the disclosure, inthe case where the processor 3 and the terminal device 5 can communicatewith each other, the setting unit 364 makes a setting such that thecommand content partly differs between the first audio command that isreceivable by the processor 3 and the second audio command that isreceivable by the terminal device 5 and accordingly it is possible toprevent an incorrect operation of the endoscope system 1.

According to the first embodiment of the disclosure, in the case wherethe processor 3 and the terminal device 5 can communicate with eachother, when the processor 3 receives inputs of the first audio commandand the second audio command from the terminal device 5, the selector367 selects a priority audio command to be prioritized, which makes itpossible to prevent an incorrect operation even when multiple audiocommands are input.

According to the first embodiment of the disclosure, when the processor3 receives inputs of the first audio command and the second audiocommand from the terminal device 5, the selector 367 selects a priorityaudio command to be prioritized based on the level of the operator ofthe endoscope 2 that is acquired by the acquisition unit 35 and thelevel of the user who uses the terminal device 5, which makes itpossible to assuredly prevent an incorrect operation.

According to the first embodiment of the disclosure, based on the levelthat is assigned to the user of the terminal device 5 on whichverification is performed by the verification unit 582, the setting unit364 sets multiple peripherals that are operable by the terminal device 5according to the second audio command, which makes it possible toprevent the peripherals from being operated by mistake.

According to the first embodiment of the disclosure, when the firstaudio command or the second audio command serves as an instruction torecord the endoscopic image data, the communication controller 363causes the communication unit 32 to transmit the endoscopic image datato the terminal device 5, which allows the operator to check thecondition of the subject, such as a patient, during the operation on theterminal device 5.

According to the first embodiment of the disclosure, the communicationcontroller 363 transmits the tag information on the content of the firstaudio command or the second audio command and the endoscopic image datain association with each other to the terminal device 5, which makes itpossible to easily search for the endoscopic image data using the taginformation after the operation on the subject.

According to the first embodiment of the disclosure, the displaycontroller 584 causes the display unit 55 to display the endoscopicimage corresponding to the endoscopic image data per set of taginformation on the content of the first audio command or the secondaudio command, which makes it possible to check a list of the group ofendoscopic images each of which is associated with each set of taginformation.

According to the first embodiment of the disclosure, the communicationcontroller 583 acquires the relevant image data that is recorded in theserver 200 based on the tag information and the display controller 584causes the display unit 55 to display the endoscopic image and therelevant image, which makes it possible to view the endoscopic image andthe relevant image in comparison with each other.

According to the first embodiment of the disclosure, the communicationcontroller 583 transmits the result of verification performed by theverification unit 582, the tag information on the content of the secondaudio command, and the endoscopic image data in association with oneanother to the server 200 that is connected with the network N100, whichmakes it possible to easily search for the endoscopic image data usingthe tag information after the operation on the subject.

Second Embodiment

A second embodiment of the disclosure will be described. In theabove-described first embodiment, the processor 3 sets peripherals thatare operable according to the audio command based on the level of theregistered user of the terminal device 5. In the second embodiment, theterminal device sets peripherals that are operable according to theaudio command based on the level of the registered user. After aconfiguration of an endoscope system according to the second embodimentis described, each of processes executed by a processor and a terminaldevice will be described. The same components as those of the endoscopesystem according to the above-described first embodiment are denoted bythe same reference numbers as those of the first embodiment anddescription thereof will be omitted.

Configuration of Endoscope System

FIG. 10 is a block diagram illustrating a functional configuration of anendoscope system according to the second embodiment of the disclosure.An endoscope system 1 a illustrated in FIG. 10 includes a processor 3 aand a terminal device 5 a instead of the processor 3 and the terminaldevice 5 of the endoscope system 1 according to the above-describedfirst embodiment.

Configuration of Processor

The processor 3 a includes a processor controller 36 a instead of theprocessor controller 36 of the processor 3 according to theabove-described first embodiment. The processor controller 36 a includesa setting unit 364 a instead of the setting unit 364 of the processorcontroller 36 according to the above-described first embodiment.

Based on an operator ID that is acquired by the acquisition unit 35 andverification information that is recorded in the verificationinformation recorder 331, the setting unit 364 a sets multipleperipherals that are operable according to a first audio command that isreceived by the audio input unit 8 via the system controller 7.

Configuration of Terminal Device

A configuration of the terminal device 5 a will be described. FIG. 11 isa block diagram illustrating a functional configuration of the terminaldevice 5 a. The terminal device 5 a illustrated in FIG. 11 includes aterminal device controller 58 a instead of the terminal devicecontroller 58 of the terminal device 5 according to the above-describedfirst embodiment. The terminal device controller 58 a further includes asetting unit 588 in addition to the configuration of the terminal devicecontroller 58 according to the above-described first embodiment.

Based on a level that is assigned to a registered user corresponding toa result of verification performed by the verification unit 582 andverification information that is recorded in the verificationinformation recorder 331, the setting unit 588 sets multiple peripheralsthat are operable according to a second audio command via the systemcontroller 7.

Process Executed by Processor

A process that is executed by the processor 3 a will be described. FIG.12 is a flowchart illustrating an overview of the process that isexecuted by the processor 3 a. The processor 3 a executes the sameprocess as the process described in the above-described first embodimentand executes step S104 a instead of step S104. Step S104 a will be thusdescribed below.

At step S104 a, based on an operator ID level that is acquired by theacquisition unit 35, the setting unit 364 a sets communicationconnection with each of the peripherals that are operable according toan audio command that is received by the audio input unit 8 via thesystem controller 7. After step S104 a, the processor 3 a moves to stepS105.

Process Executed by Terminal Device

A process executed by the terminal device 5 a will be described. Theterminal device 5 a executes a process similar to the process executedby the terminal device 5 according to the above-described firstembodiment and only the verification process differs. The verificationprocess that is executed by the terminal device 5 a will be thusdescribed below.

Verification Process

FIG. 13 is a flowchart illustrating an overview of the verificationprocess that is executed by the terminal device 5 a. In FIG. 13, stepsS901 to S903 respectively correspond to steps S801 to S803 in FIG. 8described above.

At step S904, based on the level that is assigned to the registered usercorresponding to the result of verification performed by theverification unit 582 and the verification information that is recordedin the verification information recorder 331, the setting unit 588 setseach of multiple peripherals operable according to a second audiocommand via the system controller 7.

The display controller 584 causes the display unit 55 to display a GUI,or the like, of the peripherals that are operable by the audio commandthat are set by the setting unit 588 (step S905). After step S905, theterminal device 5 a returns to the main routine in FIG. 7 describedabove.

Steps S906 and S907 correspond respectively to steps S805 and S806 inFIG. 8 described above. After step S907, the terminal device 5 a returnsto the main routine in FIG. 7.

According to the second embodiment of the disclosure described above,the setting unit 588 sets the peripherals that are operable according tothe second audio command via the system controller 7 based on the levelthat is assigned to the user of the terminal device 5 a on whichverification is performed by the verification unit 582, which makes itpossible to prevent an incorrect operation.

Other Embodiments

It is possible to form various embodiments by appropriately combiningmultiple components disclosed in the above-described embodiment of thedisclosure. For example, some components may be omitted from all thecomponents described in the embodiment of the disclosure describedabove. Components described in the embodiment of the disclosuredescribed above may be combined appropriately.

In the embodiment of the disclosure, the light source device is arrangedin the processor. Alternatively, the light source may be arrangedseparately.

The embodiment of the disclosure employs an endoscope system.Alternatively, a capsule endoscope, a video microscope that capturesimages of a subject, a mobile phone with an imaging function, or atablet terminal device with an imaging function may be employed.

The embodiment of the disclosure employs an endoscope system including amedical endoscope. Alternatively, an endoscope system including anindustrial endoscope may be employed.

In the embodiment of the disclosure, “-er” or “unit” described hereinmay be read as “means” or “circuitry”. For example, the controller maybe read as a control means or a control circuitry.

A program to be executed for the embodiment of the disclosure isrecorded in an installable or executable file data in acomputer-readable recording medium, such as a CD-ROM, a flexible disk(FD), a CD-R, a digital versatile disk (DVD), a USD medium, or a flashmemory and is provided.

The program that is executed for the embodiment of the disclosure may beconfigured such that the program is stored in a computer that isconnected to a network, such as the Internet, and is downloaded andprovided. Furthermore, the program to be executed for the embodiment ofthe disclosure may be configured to be provided or distributed via anetwork, such as the Internet.

In the description of the flowcharts herein, the context of the processamong steps is clearly specified using expressions including “first ofall”, “then”, or “subsequently”; however, the order of the processesnecessary to implement the disclosure is not uniquely determined bythose expressions. In other words, the process order of the flowchartsdescribed herein is changeable within a range causing no inconsistency.The embodiment is not limited to the program including simple branches,and the program may be branched off according to general determinationon more determination items. In that case, the technology of artificialintelligence that performs machine learning while repeating learning byinducing the user to perform manual operations may be used together. Theartificial intelligence may be implemented in a manner that theartificial intelligence is caused to learn operation patterns performedby many specialists and perform deep learning such that more complicatedconditions are incorporated.

The disclosure achieves an effect that, even when multiple audio inputunits are used, it is possible to prevent an incorrect operation.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the disclosure in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An endoscope system comprising: a processorconfigured to perform image processing on endoscopic image data that isacquired by an endoscope that performs in-vivo observation on a subject,the processor being connectable with multiple peripherals, the processorbeing configured to receive an input of a first audio command serving asan instruction to drive any one of the peripherals based on audio datathat is generated by an audio input circuit receiving an input of sound;a terminal device configured to wirelessly communicate with theprocessor and receive an input of a second audio command serving as aninstruction to drive any one of the peripherals using sound; a settingcircuit configured to, in a case where the processor and the terminaldevice can communicate with each other, make a setting such that thefirst audio command that is receivable by the processor and the secondaudio command that is receivable by the terminal device differ from eachother at least partly; and a first communication controller configuredto, when the first audio command or the second audio command serves asan instruction to record the endoscopic image data, transmit theendoscopic image data to the terminal device.
 2. The endoscope systemaccording to claim 1, further comprising: a selector configured toselect a priority audio command to be prioritized when the processorreceives inputs of the first audio command and the second audio commandfrom the terminal device in the case where the processor and theterminal device can communicate with each other; and a drive controllerconfigured to drive any one of the peripherals based on the priorityaudio command that is selected by the selector.
 3. The endoscope systemaccording to claim 2, further comprising: an acquisition circuitconfigured to acquire operator identification information thatidentifies an operator of the endoscope; and a recorder configured torecord multiple sets of the operator identification information inassociation with levels according to which operating each of theperipherals is permitted, wherein the selector is configured to, whenthe processor receives the inputs of the first audio command and thesecond audio command from the terminal device, select the priority audiocommand based on a level of the operator that is acquired by theacquisition circuit and on a level of a user who uses the terminaldevice.
 4. The endoscope system according to claim 1, further comprisinga system controller that is connected to the processor and each of theperipherals, the system controller being configured to control drivingthe peripherals, wherein the terminal device further includes averification circuit configured to verify whether a user of the terminaldevice is a registered user who is registered previously, and theprocessor includes a drive controller configured to drive any one of theperipherals based on the first audio command or the second audiocommand; and the setting circuit, the setting circuit is configured to,based on a level that is assigned to the user on which verification isperformed by the verification circuit, set the peripherals that areoperable by the terminal device according to the second audio commandvia the system controller, and the drive controller is configured todisable the second audio command that is input from the terminal devicewhen the setting circuit does not make a setting such that theperipherals corresponding to the second audio command that is input fromthe terminal device are operable.
 5. The endoscope system according toclaim 1, further comprising a system controller that is connected to theprocessor and each of the peripherals, the system controller beingconfigured to control driving the peripherals, wherein the terminaldevice further includes a verification circuit configured to verifywhether a user of the terminal device is a registered user who isregistered previously; and the setting circuit, and the setting circuitis configured to, based on a level that is assigned to the user on whichverification is performed by the verification circuit, set theperipherals that are operable according to the second audio command viathe system controller.
 6. The endoscope system according to claim 1,wherein the processor includes the first communication controller, andthe first communication controller is configured to transmit, to theterminal device, tag information on content of the first audio commandor the second audio command and the endoscopic image data in associationwith each other.
 7. The endoscope system according to claim 6, whereinthe terminal device further includes a display configured to display animage; and a display controller configured to control a display mode ofthe display, and the display controller is configured to cause thedisplay to display an endoscopic image corresponding to the endoscopicimage data for each set of the tag information.
 8. The endoscope systemaccording to claim 7, further comprising a server that is connected tothe terminal device and the processor via a network, the server beingconfigured to record relevant image data that is relevant to theendoscopic image, wherein the terminal device further includes a secondcommunication controller configured to acquire the relevant image datathat is recorded in the server based on the tag information, and thedisplay controller is configured to cause the display to display theendoscopic image and a relevant image corresponding to the relevantimage data.
 9. The endoscope system according to claim 1, wherein theterminal device further includes a record controller configured torecord the second audio command of which input is received by theterminal device and the endoscopic image data in association with eachother in a recorder.
 10. The endoscope system according to claim 4,wherein the terminal device further includes a second communicationcontroller configured to transmit a result of verification performed bythe verification circuit, tag information on content of the second audiocommand, and the endoscopic image data in association with each other toa server that is connected to a network.
 11. The endoscope systemaccording to claim 1, wherein the first audio command is configured of acommand according to which all the peripherals are operable, and thesecond audio command is configured of a command according to which aperipheral not directly relevant to an operation is operable.
 12. Acontrol method that is executed by an endoscope system including aprocessor configured to perform image processing on endoscopic imagedata that is acquired by an endoscope that performs in-vivo observationon a subject, the processor being connectable with multiple peripherals,the processor being configured to receive an input of a first audiocommand serving as an instruction to drive any one of the peripheralsbased on audio data that is generated by an audio input circuitreceiving an input of sound; and a terminal device configured towirelessly communicate with the processor and receive an input of asecond audio command serving as an instruction to drive any one of theperipherals using sound, the control method comprising: in a case wherethe processor and the terminal device can communicate with each other,making a setting such that the first audio command that is receivable bythe processor and the second audio command that is receivable by theterminal device differ from each other at least partly; and when thefirst audio command or the second audio command serves as an instructionto record the endoscopic image data, transmit the endoscopic image datato the terminal device.